Code by Octave

function [jVal, gradient] = costFunction(theta)

    jVal = (theta(1) - 5)^2 + (theta(2) - 5)^2;
    gradient = zeros(2, 1);
    gradient(1) = 2*(theta(1) - 5);
    gradient(2) = 2*(theta(2) - 5);
end

優化算法可自動選擇學習函數alpha的gradient descent。

options = optimset('GraObj', 'on', 'MaxIter', '100');
initialTheta = zeros(2, 1);
[optTheta, functionVal, exitFlag] = fminunc(@costFunction, initialTheta, options);

1.Underfitting(High Bias)
2.Just right
3.Overfitting(High Variance)

Addressing overfitting

1. Reduce number of features(Manually and Model selection algorithm)
2. Regularization(保留所有特徵值，降低theta以避免overfitting

1/2m * ∑(h(x(i)) - y(i))^2 + lambda * ∑((theta)^2), i = 1:m; j = 1:n